Energy Demand Forecasting Model Performance Report

The following report contains model performance metrics for the NY City Hourly Probabilistic Residential Energy Demand Forecasting Pipeline. Model performance was evaluated on both long-term and day-ahead forecasts. Evaluation was conducted using a holdout dataset of hourly energy demand values between 2023-10-19 and 2024-05-23.

Long-Term Forecasting Performance

The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
MSE Weighted MSE MAE MAPE Wilcoxon Test p-value
Forecasting Pipeline 132582.09 117777.65 274.49 0.05 NaN
Baseline Yearly MA 618450.20 618450.20 635.02 0.14 <0.01

The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand values. It also presents the 95% confidence interval bounds estimated by the forecasting model.



Day-Ahead Forecasting Performance

The following table contains performance metrics for the forecasting model compared with a yearly moving average baseline model.
MSE Weighted MSE MAE MAPE Wilcoxon Test p-value
Forecasting Pipeline 908700.32 908575.29 820.12 0.16 NaN
Baseline Moving Avg 350660.80 350660.80 512.26 0.10 1.0
EIA Forecasts 115596.45 NaN 291.92 0.06 1.0

The following Plotly Figure helps to contextualize the forecasting model's performance by showing its predictions along with the actual energy demand values. It also presents the 95% confidence interval bounds estimated by the forecasting model.



Model Sensitivity Analysis

The following figure shows a permutation feature importance analysis.

Sensitivity Analysis